Method and system for determining whether a spherical element impacts with a component of a playing field, or arranged on or proximate thereto

ABSTRACT

A method and system for determining whether a spherical element impacts with a component of a playing field, or arranged on or proximate thereto. The method includes acquiring images of a surveillance area of a field that covers at least part of said component, such as a delimiting perimeter line of a game area or a target, performing an approximate detection of an impact of a spherical element relative to that component or proximate thereto, with an object detection and recognition system that can discern when the detected object is indeed a spherical element, automatically selecting one of the images acquired for the same point in time and that includes the area where said impact has occurred, and analyzing the selected image to check if the spherical element has impacted or not with the component.

TECHNICAL FIELD

The present invention generally concerns, in a first aspect, to a methodfor determining whether a spherical element impacts with a component ofa playing field, or arranged on or proximate thereto, by performing anapproximate detection and a more refined detection comprising theautomatically selection, in response to the approximate detection, ofone or more surveillance images, and more particularly to a methodwherein the approximate detection is performed by a system which candiscern whether the detected object is indeed a spherical element.

A preferred application of the invention relates to judging whether thespherical element bounces inside or outside game areas.

A second aspect of the invention concerns a system suitable forimplementing the method of the first aspect.

The invention preferably applies to a tennis match, where the game areasare the different rectangular areas of a tennis court, the sphericalelement is a tennis ball and according to the rules, the ball isconsidered to bounce within the game area if it touches any of theperimeter lines that delimit the area.

The invention also applies to the training of tennis players, to improvethe accuracy of their shots, by the arrangement of targets on which theball must bounce after hitting it.

STATE OF THE ART

When playing tennis, many balls bounce so fast on the lines delimitingthe different game areas of the court, i.e. “in” or “out” of the courtor the serving area, that the human eye is not able to follow them andthey leave no trace, which becomes a source of problems.

There exist numerous proposals the purpose of which is to judge whethera spherical element, in particular a tennis ball, has bounced in or outof a game area.

The best known proposal, due to the adoption of its use in officialtennis matches, is implemented by the system known as “Hawkeye”, inwhich a simulation of the ball trajectory captured by costly high-speedcameras is reproduced, and allowing to see if the ball is “in” or “out”in theory with an accuracy of 3 mm.

This system has many drawbacks: it is very expensive (16 very costlycameras), it requires other people to watch the images and select thecorrect ones, and furthermore, the trajectory simulation has anestimated error margin of 3 mm, which many people consider to be in factmore than that value. The player does not see a real representation, itis a “leap of faith”, since the images are simulated.

There are also proposals where the images analysed are not simulated,but real. Some patent documents relating to such proposals are citedbelow.

U.S. Pat. No. 8,199,199B1 provides a method and a system forregistration of real time position of tennis ball relatively to theboundary lines of game areas on a tennis court, and which comprisescapturing images of the court using at least one video camera sensitiveto near-infrared radiation, and illuminating the court withnear-infrared radiation from a light source displaced relatively to thevideo camera, in order to determine the trajectories of the tennis balland its shadow, determining the point of intersection and comparing theposition with previously calibrated boundary lines of court as a way ofdetermining if a tennis ball bounces in or out of the game area.

The system proposed in said patent is overly complex since it mustperform the detection and tracking of two moving elements, the ball andits shadow, and although a filtering of the captured images is proposedto avoid confusing these two elements with other moving elements presentin the field of view of the video camera, the results of this filter aredoubtful, especially with regard to the shadows of other objects thatmay interfere with the shadow of the ball.

Furthermore, the efficiency of the implementation of such a system isalso very doubtful, since, especially as regards the shadow of the ball,this will vary depending on the points where the video camera or camerasare located, and the light source, and in order to appreciate it clearlythe video camera must be located at a certain height, which makes itdifficult to ensure that at the point of intersection of the ball andits shadow the ball has in fact made contact with the court or, even ifit does, the precise point of the court it touched.

From international application WO2007098537A1 a method and system isknown that contains, respectively, the features of the preamble of theindependent claims of the present invention. The method and systemproposed in WO2007098537A1 aims to determine parameters of contactbetween bodies, such as the bounce of a spherical element on a playingfield, by means of performing an approximate detection involving thedetection of localized infrared emissions owing to heat generated by thefriction between the bodies or the deformation of at least one of thebodies, and the automatic selection and subsequent display of an imageor images associated with this approximate detection and which includesdetails of the scene where this possible contact occurred, in theinfrared field and preferably also in the visible field, to enable avisual inspection to determine, for example, if the spherical elementhas bounced in or out of a game area.

The system used to perform the aforementioned approximate detection inWO2007098537A1, i.e. detecting infrared emissions, it is veryinefficient, since the infrared emission can be produced for variousunwanted causes other than the contact between the bodies of interest,i.e. that are not associated, for example, with the contact between thespherical element and the sports court. Such causes can be diverse,ranging from the contact caused by a body that is not of interest, suchas a leaf falling on the court or the player himself moving through it,to sudden changes in temperature in the area of interest, for exampleowing to the presence of areas of sun and shade therein, or because thetennis court is an outdoor court subjected to high temperatures.

An earlier and less efficient detection system than those mentionedabove, and which is not based on images, is described in patentGB2292218, which proposes a method and apparatus for judging whether atennis ball has bounced on a line using an array of photo-detectors uponwhich a single collimated laser beam strikes, detecting the presence andtrajectory of the ball according to the photo-detectors in front ofwhich the ball passes, i.e. a system that could be considered as analternative to image acquisition system proposed in WO2007098537A1, andclearly outperformed by the latter in terms of detection efficiency.

Patent GB2292218 does not describe, or even hint, that the object to bedetected may not be a tennis ball, therefore no solution is proposed tothis problem completely ignored by GB2292218: neither an approximatedetection system, as proposed by WO2007098537A1, nor any other kind ofmechanism auxiliary to the photo-detection system. It should be notedthat such a photo-detection system, among other disadvantages, such asfor example misinterpreting as a bounce on the line the crossing of thebeam through an area that is not on the line (both the emitter andreceiver are separated by a certain distance from the ends of the lineto be monitored), it has the disadvantage that it is clearly unable toperform three-dimensional detection, i.e. it only performs atwo-dimensional detection which makes it obviously impossible to detecta sphere, i.e. a three-dimensional object, at most being capable ofdetecting whether the object projects, in any of its positions, atwo-dimensional circular silhouette which could happen even though theobject is not spherical.

On the other hand, several different proposals are known concerningtraining systems providing one or several targets on the tennis court inorder for the player tries to hit the ball so that it bounces on theselected target, and that detect, with greater or lesser precision, inwhat areas of the tennis court the ball impacts after the player hitsit.

One such proposal is described in US2008293522A1, which provides afreely positionable tennis court practice target as well as a displaythat responds to a signal generated by the target (for example,wirelessly) and provides an indication when each target area isimpacted. The display shows the number of impacts in each area, andlight and/or sound indicators indicate when the impact occurred in eacharea. The target is a mat with areas sensitive to impacts, configuredfor example by piezoelectric elements or other sensors.

Patent ES2421399T3 proposes one of such target mats to detect theimpacts of balls.

The document of patent TW201135216A proposes detecting the areas ofimpact of the ball on, in this case, a ping pong table, by using anarray of laser beam transmitters and receivers.

Moreover, international application WO2013124856A1 concerns a system formonitoring and analysing moving objects applicable, among others, totennis, consisting in marking the different positions of a virtualopponent on the tennis court using a laser pointer or with a hologram,automatically following predefined training programs or simulations ofthe virtual opponent. The detection of where the tennis ball impacts,i.e. whether it does on the virtual opponent or the mark thereof, isperformed by a system based solely on cameras.

All the training systems cited that provide one or more targets on thetennis court can clearly be improved in terms of accuracy in detectingthe impact of the ball on target.

SUMMARY OF THE INVENTION

It is necessary to provide an alternative to the prior art which willcover the voids found therein and, in particular, that offers increasedperformance and efficiency, in terms of the percentage of successes orguesses, clearly superior to those obtained with the method and systemproposed by WO2007098537A1, and which moreover also clearly improves thedetection accuracy of the impact of a spherical element on a target inrelation to known systems and methods.

To this end, the present invention relates, in a first aspect, to amethod for determining whether a spherical element impacts with acomponent of a playing field, or arranged on or proximate thereto, wheresaid field is formed by a number of game areas, wherein the methodcomprises:

a) acquiring a sequence of images of at least one area of surveillanceof said playing field covering at least part of at least said component;

b) performing an approximate detection of the impact of a sphericalelement with said component or proximate thereto;

c) automatically selecting, in response to said approximate detection,at least one image, from said sequence of images, acquired for the samepoint in time and that includes the area where said impact took place;and

d) analysing the at least one selected image to check if the sphericalelement has really impacted or not with said component.

Unlike the known proposals, the method proposed by the first aspect ofthe present invention comprises, characteristically, performing theaforementioned approximate detection with an object detection andrecognition system that can discern automatically when the objectdetected is indeed a spherical element, for which purpose, obviously,the object detection and recognition system is of a three-dimensionaltype.

According to a preferred embodiment, the component of the playing fieldis one of the perimeter lines among those perimeter lines that delimitthe game areas, and the method is intended for judging whether aspherical element bounces in or out of these game areas, with the bounceconstituting said impact, where:

said step a) comprises acquiring a sequence of images from at least onearea of surveillance of the playing field that covers at least part ofat least one of said perimeter lines, generally consisting of one ormore areas of surveillance, where together they cover all the perimeterlines of the game areas;

said step b) comprises performing an approximate detection of the bounceof a spherical element on one of said perimeter lines or proximatethereto;

said step c) comprises automatically selecting, in response to saidapproximate detection, at least one image, from said sequence of images,acquired for the same point in time and that includes the area wheresaid bounce took place; and

said step d) comprises analysing the at least one selected image, tocheck if the spherical element has really bounced or not on saidperimeter line, in order to judge if the spherical element has bouncedinside or outside the game area delimited by said perimeter line of stepb)

According to another embodiment (in general alternative to the above butwhich also could be combined therewith), the component of said playingfield is a target arranged inside at least one of said game areas, whichare delimited by a perimeter lines, wherein:

said step a) comprises acquiring a sequence of images of at least onearea of surveillance of said playing field covering at least part ofsaid target;

said step b) comprises performing an approximate detection of a bounceof a spherical element on said at least one target, or proximatethereto;

said step c) comprises automatically selecting, in response to saidapproximate detection, at least one image from said sequence of imagesacquired for the same point in time and that includes the area wheresaid bounce took place; and

said step d) comprises analysing the selected image, of which there mustbe at least one, to check if the spherical element has really bounced ornot on the target of step b).

According to a preferred variation of this embodiment example where thecomponent is a target, the latter is constituted by a sub-area of atleast one of the game areas and is marked physically or virtually on theground, projected on the ground (e.g. using a laser pointer) or ondisplaying means visible to the player, for example on a large screenplaced on the playing field.

Less preferably, the target does not consist of a sub-area of the gamearea but instead placed thereupon, not necessarily at ground level,physically or virtually (as in the case of a hologram).

Continuing with this embodiment, the method comprises, optionally,modifying the shape, size and/or position of the target, with referenceto the playing field, and performing the steps a) to d) to detectwhether or not the spherical element has bounced on the modified target.

Alternatively, or in addition, the method comprises, after completion ofstep d) for a target, proceeding to select another target and performingsteps a) to d) to detect whether or not the spherical element hasbounced on said another target.

For the embodiment in which the method of the first aspect of theinvention is applied to determine whether spherical element bounces on atarget, this is preferably associated with a training method or systemthat performs the aforementioned selection and/or modification oradaptation of the targets, as well as their scoring and repetition, atthe choice of the player or coach, for example remotely through a mobilecomputing device (such as a “tablet” or “smartphone”) or automaticallyfollowing a training pre-programmed program, and that measures theresults obtained by the player, depending on his aim when hitting thespherical element at the targets. All the points, areas (i.e. targets ordifferent target areas, etc.), can be viewed live on a screen on theplaying field and/or controlled through said mobile computing device.

Preferably said training system includes a mobile robot or machinewhich, if the playing field is a tennis court, moves along the courtlaunching balls based on a configured series or in relation to theposition of player, which is identified by the object detection andrecognition system itself.

This training system optionally includes other kinds of measuringdevices, e.g. speed and/or power of the ball hit by the player, therebyproviding not only the precision of the stroke as a result, but alsopower and/or speed measurements.

Although this application has not claimed such a training method orsystem which is associated with the method and/or system of the presentinvention, the description included here is sufficient for supportingand seeking the independent protection of such a training method andsystem, for example through a divisional application.

The method proposed by the first aspect of the invention method is alsoapplicable to the detection of the impact of a spherical element with acomponent that is neither of the two explained above, i.e. that is not aperimeter line or a target, like for example, if the playing field is atennis court, a post (near the playing field, such is case of the postsholding up the centre net), the centre net (both the portion includedwithin the field of play delimited by the perimeter lines and theportion that is not, i.e., the area close to the field of play, etc.).For other kinds of playing fields, the components can be of variouskinds, such as, for example, a basketball court, the rim of the basketor a player, etc.

For one embodiment, the method comprises using a three-dimensionalscanner as the detection and recognition system, active or passive,employing any of those known in the prior art of the “non-contact” type,i.e. those which work by analysing the return signal (emitted expresslyin the case of active systems, or the environment's own signal in thecase of passive systems) to capture the geometry of an object or scene.Electromagnetic waves can be used (from radio waves to X-rays) orultrasound which may be combined with different devices for transmissionand detection, such as a laser emitter (for example structured light ormodulated light) and a detector comprising a camera that detects thereflected light.

For a preferred embodiment, this three-dimensional scanner system is anactive laser scanner system, such as a LIDAR system (acronym for “LightDetection and Ranging” or “Laser Imaging Detection and Ranging”) thatincludes one or more laser scanners.

Alternatively, the three-dimensional scanner system consists of aplurality of cameras operating synchronously and spatially arranged soas to acquire three-dimensional information, and that are thus capableof detecting and recognizing three-dimensional objects, and thereforeautomatically discern when the detected object is indeed a sphericalelement.

Depending on the embodiment, the aforementioned approximate detection instep b) comprises performing two or more detections including at least amoment when said impact or bounce happens and/or at least a momentimmediately before the impact or bounce and/or at least a momentimmediately after the impact or bounce, using in said step c), as a timepoint for the automatic selection of said image, at least one of thetime points of said detections, preferably the one corresponding to theimpact or bounce or the moment immediately before.

For the embodiments where the component on which to detect the impact ofthe spherical element is located at ground level, as is the case of theperimeter line, and, in general, of the targets, such detection isperformed approximately for the same height or close to the ground, inorder to detect the spherical element when it bounces thereon and whenit is about to or has just bounced and according to an embodiment usingtwo respective laser scanners.

If the component is not located at ground level, such as the centre netof a tennis court, the approximate detection is performed at a suitableheight for this component.

According to one embodiment, the method comprises, prior to theapproximate detection and, in general, at greater height thereof,performing a pre-detection of the passage of the spherical elementthrough at least two points located at different heights, andperforming, based on said pre-detection, a prediction of the trajectoryto be followed by the spherical element towards the ground and apredetermination or predefinition of an approximate area where theimpact or bounce will occur.

For an embodiment where the approximate detection is not implemented atground level, as in the case discussed above in which the component isthe centre net of a tennis court, at least one pre-detection like theone described takes place at a lower height than the approximatedetection, in order to predict the upward trajectory of the sphericalelement, as would be the case where a tennis player hits a ball fromdown going up in order to get over the centre net. This embodiment is analternative (in the case of wanting to cover only an upward trajectory)or complementary (in the case of wanting to cover both trajectories: theone going up and going down) to the previous embodiment.

Said predetermination or predefinition of the approximate area where theimpact or bounce of the spherical element will occur, obtained asdescribed in the preceding paragraphs or through any other kind ofmechanism considered suitable by an expert in the matter, in itselfrepresents an alternative to the approximate detection of step b). Thatis to say, according to another aspect of the present invention,alternative to the first aspect, the present invention concerns a methodfor determining whether a spherical element impacts with a component ofa playing field, or arranged on or proximate thereto, said field formedby game areas, wherein the method comprises:

a) acquiring a sequence of images of at least one area of surveillanceof said playing field covering at least part of at least said component;

b) performing a predetermination or predefinition of the approximatearea where the impact or bounce of the spherical element will occur;

c) automatically selecting, in response to said predetermination orpredefinition of an approximate area, at least one image from saidsequence of images acquired for the same point in time as the expectedimpact or bounce, and that includes the area where said impact or bounceis expected to take place; and

d) analysing the selected image, of which there must be at least one, tocheck if the spherical element has really impacted, or bounced, or notwith said component.

The method proposed by this alternative to the first aspect of thepresent invention comprises, preferably, performing the aforementionedapproximate detection with an object detection and recognition systemthat can discern automatically when the object detected is indeed aspherical element, for which, obviously, the object detection andrecognition system must be of a three-dimensional type.

The method comprises carrying out such pre-detection using at least twodetection devices, such as two laser scanners from the object detectionand recognition system, located at the two aforementioned points atdifferent heights.

The method comprises automatically discerning when the detected objectis indeed a spherical element based on the difference in the detectionsignals caused by being detected by the detection and recognition systemin comparison with the detection signals resulting from the detection ofother elements, stationary or mobile, of at least different shape andsize.

The detection and recognition system “knows” where the feet of theplayers or any other element of the court are located, allowing it tomake the distinction between the spherical element and these elements.Subsequently, when the system proposed by the second aspect of theinvention is explained, there will be a more detailed description of thesystem components and how they operate to perform the aforementionedautomatic differentiation and detection of the spherical element fromother elements. The description below of the operation of such elementsshall also be considered valid for defining the actions performedaccording to the method proposed by the first aspect of the invention,for some embodiments.

For other embodiments, the method comprises using as the detection andrecognition system, alternatively or complementary to the laser scannersystem, other types of detectors such as linear detectors (in this casein a complementary manner to a suitable system for capturingthree-dimensional information) or cameras, or other types of scanningsystems, like those indicated above.

For a preferred embodiment, said spherical element is a tennis ball andthe playing field is a tennis court, where the game areas are thedifferent rectangular areas of the tennis court, so that the methoddetermines whether the ball has bounced in or out of the court orwhether it has bounced inside or outside the rectangular area used forthe service.

Continuing with this preferred embodiment applied to tennis, accordingto a variation thereof, the method proposed by the first aspect of thepresent invention comprises selecting as perimeter lines on which checkif the tennis ball has really bounced or not, the following:

-   -   those demarcating a respective service area (left or right) to        which a player will serve or has served, to judge whether the        tennis ball has bounced in or out of the service area; or    -   those demarcating the total game area for a single or a doubles        match during the rally after the service.

Additionally, the method comprises receiving information on whether,after the service, the tennis ball has touched the net or not, and basedon this information, after judging whether the tennis ball bounces in orout of the service area, the method comprises issuing a signalindicating that:

-   -   the serve must be repeated, if the ball has bounced inside the        service area but after touching the net, or    -   the serve must not be repeated, if the ball has bounced outside        the service area after touching the net.

The detection of whether or not the ball has touched the net can beperformed with a sensor arranged on the net itself or even by the objectdetection and recognition used by means of the method proposed by thefirst aspect of the present invention.

Depending on the type of game played, if a spherical element has touchedthe perimeter line it will be considered to have bounced inside the gamearea, such is the preferred case applied to tennis, or out of the gamearea, as would be the case, less preferred, applied to basketball.

According to one embodiment, step d) comprises showing the selectedimage or images on displaying means to allow performing theaforementioned analysis by visually judging the image, or to supplementsaid analysis.

For one embodiment, the analysis of step d) is an automatic analysis,performed by image processing programs or algorithms.

Depending on the embodiment, the method comprises initiating step d)upon request of a player or umpire, or automatically after the selectionof step c).

In the first case, the request of the player or umpire may require theintervention of a third party, such as an operator, to initiate the stepd) upon receipt of the request, by gesture or voice, by the player orumpire, or automatically, for example by implementing a control systemthat recognises the gesture of the player or umpire and automaticallyactivates the start of step d).

Such gesture recognition and control systems are known, ranging fromthose based on the analysis of images that capture these gestures tomore sophisticated systems that include a movement detector device wornby the player or umpire, such as an inertial and/or electromyographicwristband that can recognize the gesture performed with the wrist and/orhand and/or arm by the player and/or umpire.

According to one embodiment, step c) of the method proposed by the firstaspect of the invention comprises selecting a plurality of images,arranged in sequence, which includes the image acquired for the samepoint in time as the approximate detection and images for points in timebefore and after that point.

The aforementioned automatic analysis is, according to one embodiment,relative to said plurality of selected images, and includes at least thechoice of the image where the spherical element appears most deformed incontact with ground, and/or with said component, and the subsequentautomatic analysis thereof and/or display thereof on said displayingmeans.

For one embodiment, step d) comprises a video playback including saidplurality of images, in slow motion forward and/or backwards and/orstopping in order to perform the aforementioned visual judgment.

For another embodiment, complementary or alternative to the above, stepd) comprises displaying a magnified area of interest of the selectedimage or images centred on the area of impact or bounce of the sphericalelement.

The method comprises performing step a), according to a preferredembodiment, by means of a plurality of high speed cameras covering thevarious components of the playing field, or arranged on or proximatethereto, and/or different sections of the perimeter lines from bothsides and/or ends thereof and/or all of the targets from different sidesthereof.

According to one embodiment, both in the first aspect and in theaforementioned alternative aspect to the first aspect, the methodcomprises using the above-described predefinition of an approximate areawhere the impact or bounce will occur to activate (if not alreadyactivated) and/or control at least one of said high-speed cameras whosecoverage area includes the approximate predefined area, this controlbeing relative to, for example, the performance of a micro-focus of theactivated camera in order to focus the image acquisition area to thearea of interest.

A second aspect of the invention relates to a system for determiningwhether a spherical element impacts with a component of a playing field,or arranged on or proximate thereto, where said field is formed by anumber of game areas, wherein the method comprises:

-   -   means for acquiring and recording images configured and arranged        to acquire and record a sequence of images of at least one        surveillance area of said playing field that covers at least        part of said component;    -   means of detection configured and arranged to perform an        approximate detection of an impact of a spherical element with        said component or proximate thereto;    -   means of automatic selection connected to said means of        detection, with access to said recorded images, and configured        to automatically select, in response to said approximate        detection, at least one image of said sequence of images        acquired for the same point in time and which includes the area        where said impact has been produced; and    -   means of displaying and/or of analysis configured for,        respectively, displaying said at least one selected image, to        allow it to be judged by visual analysis, and/or to analyse it        automatically, to check if the spherical element has really        impacted or not with said component.

Unlike known systems, in the system proposed by the second aspect of thepresent invention, characteristically, the detection means comprise anobject detection and recognition system that can discern automaticallywhen the object detected is indeed a spherical element.

According to a preferred embodiment, the component of the playing fieldis one of the perimeter lines among those perimeter lines that delimitthe game areas, and the system is provided for judging whether aspherical element bounces in or out of these game areas, with the bounceconstituting said impact, where:

-   -   said means for acquiring and recording images are configured and        arranged to acquire and record a sequence of images of at least        one surveillance area of said playing field that covers at least        part of at least one of said perimeter lines;    -   said means of detection are configured and arranged to perform        an approximate detection of the bounce of a spherical element on        one of said perimeter lines or proximate thereto;    -   said means of automatic selection connected to said means of        detection, with access to said recorded images, and configured        to automatically select, in response to said approximate        detection, at least one image, of said sequence of images,        acquired for the same point in time and which includes the area        where said bounce has been produced; and    -   said means of displaying and/or of analysis are configured for,        respectively, displaying said at least one selected image, to        allow it to be judged by visual analysis, and/or to analyse it        automatically, to check if the spherical element has really        bounced or not on said perimeter line of said approximate        detection, in order to judge if the spherical element has        bounced inside or outside the game area delimited by said        perimeter line.

According to another embodiment (in general alternative to the above butwhich also could be combined therewith), the component of said playingfield is a target arranged inside at least one of said game areas, whichare delimited by perimeter lines, wherein:

-   -   said means for acquiring and recording images are configured and        arranged to acquire and record a sequence of images of at least        one surveillance area of said playing field that covers at least        part of at least said target;    -   said means of detection are configured and arranged to perform        an approximate detection of a bounce of a spherical element on        said at least one target, or proximate thereto;    -   said means of automatic selection are connected to said means of        detection, with access to said recorded images, and configured        to automatically select, in response to said approximate        detection, at least one image, of said sequence of images,        acquired for the same point in time and which includes the area        where said bounce has been produced; and    -   said means of displaying and/or analysis are configured for,        respectively, displaying said at least one selected image, to        allow it to be judged by visual analysis, and/or to analyse it        automatically, to check if the spherical element has really        bounced or not on the target in step b).

According to a preferred variation of this embodiment where thecomponent is a target, the latter is constituted by a sub-area of leastone of the game areas, whereby the aforementioned means of displayingare also configured to display said target by means of an image thereoftaken by at least one camera, when the target is marked physically onthe ground, or by means of a virtual representation thereof, when thetarget is only marked virtually, projected on the ground or viadisplaying means visible to the player.

Preferably, the spherical element is a tennis ball and the playing fieldis a tennis court, with the game areas being the different rectangularareas of the tennis court.

For one embodiment, the detection and recognition system includes anythree-dimensional scanning system, active or passive, of those describedabove with reference to the method proposed by the first aspect of theinvention, or any known system.

According to a preferred embodiment, the detection and recognitionsystem includes at least one laser scanner system which comprises one ormore laser scanners.

According to one embodiment, the detection and recognition systemcomprises, in order to perform the aforementioned approximate detection,at least a first detection device (such as a laser scanner) arranged atground level or proximate thereto, with its field of coverage includingat least a portion of the component and/or of the perimeter line and/orof the target, on or near to which said impact or bounce occurs.

In one embodiment, the detection and recognition system comprises atleast a second and a third detection device (such as two laser scanners)located in two respective points at different heights, both higher thanthe first detection device, arranged and configured to detect thepassage of a moving element through at least said two points, thedetection and recognition system being configured to perform, based onthe detection of the passage through said two points, a prediction ofthe trajectory to be followed by the moving element, and to discernwhether or not it is a spherical element.

When the moving element whose passage has been detected by said twopoints is the spherical item, the aforementioned prediction of histrajectory also includes the previously described (with reference to themethod of the first aspect of the invention) prediction andpredefinition of an approximate area where the bounce will occur.

The detection and recognition system can automatically discern when theobject detected is indeed a spherical element because it includes aprocessing system that implements a complex algorithm with access todetection information of all the detection devices listed above, bothpreviously recorded for the stationary elements located within the areasof coverage of the detection devices, and those corresponding to currentdetection of moving elements, first through the second and thirddetection devices described above, to detect the passage of a movingelement and its trajectory, and then with the detection device arrangedat ground level, allowing to check, for example, if the moving elementwas a person's foot.

The only moving element with the dimensions of the spherical element isthe spherical element itself, whose detection by the detection deviceproduces signals that are perfectly distinguishable from the signalsproduced by other elements.

For the preferred embodiment where the detection devices are laserscanners, these allow distance information to be obtained with respectthereto from each point of contact of the laser beam with the detecteditem, with the grouping and number of contact points being verydifferent when the element is a spherical element or, for example, aplayer's foot.

Advantageously, the detection and recognition system comprises aplurality of the first, second and third detection devices arranged tocover different areas of, and above, the game areas, which allows, inthe case where these are laser scanners, to calculate the position ofeach object in the game areas at all times.

For one embodiment, the system comprises a control means connected tothe automatic selection means and with said means of displaying and/orof analysis, to control the latter so that they display (preferablyenlarged) and/or analyse the selected image or images, e.g. on a screenlocated on the court or any other type of display device.

The means for acquiring and recording images of the system proposed bythe second aspect of the invention comprises, according to oneembodiment, a plurality of high speed cameras (at least in the order of1000 frames per second) arranged so that their coverage areas cover thevarious components of said playing field, or arranged on or proximatethereto, from different sides of the components, and/or differentsections of the perimeter lines from both sides and/or ends thereofand/or all the targets from different sides thereof, such that in theselection of the image performed by the automatic selection means, basedon information of the area where the impact or bounce of the ball hasbeen approximately detected, the high-speed camera monitoring that areais selected, in turn selecting the image captured by that camera for theexact millisecond the approximate detection occurred.

These high speed cameras are preferably arranged at ground level, orsubstantially at ground level, so that they exactly capture the contactsurface of the ball when it is deformed on the ground, which, togetherwith the large number of images captured per second by the cameras,allows the image where the ball appears most distorted to be chosen foranalysis, without any kind of simulation, whereby the accuracy of thesystem is very high.

According to one embodiment, at least one of the high-speed cameras oranother camera of the means for acquiring and recording images,preferably two, is mobile.

The memory or memories where the images are recorded are part of theelectronic circuitry of the cameras themselves or of an externalelectronic system.

According to a variant of this embodiment the mobile cameras and/or TOFcameras (“Time Of Flight”) are mounted on intelligent mobile robots that“understand” the circumstances of the game. Each of these robotsincludes a supervision, tracking and prediction system that makes itaware of the circumstances of the game, including the position at alltimes of the players and the spherical element, and that predicts, bycalculating, where the spherical element is going to impact or bounce,making the robot suitable for, based on such knowledge and prediction,finding the best angle of capture of the mobile camera and moving in thedirection of the area where the spherical element is expected to bounce,allowing an image of the most appropriate area of interest to beobtained and therefore a more accurate calculation.

It is also possible, according to another embodiment, to use theabove-described predefinition of an approximate area where the bouncewill occur in order to control the robot and direct it thereto for thepurpose of obtaining detailed images of the bounce of the sphericalelement.

The system of the second aspect is adapted to implement the method ofthe first aspect, performing the steps of automatic selection andautomatic image analysis generally by means of software installed in acomputer system that implements the selection and analysis means. Thedescription of all the embodiments of the method proposed by the firstaspect of the invention are therefore valid for describing analogousembodiments relating to the system proposed by the second aspect of theinvention, the latter incorporating appropriate means for performing thefunctions described in relation to the method, including the meansalready described above or additional thereto.

The aforementioned software installed on a computer system thatimplements the selection and analysis means therefore allows the liveand near real-time analysis of whether the ball is out or not (theinformation is preferably transmitted in less than a second), by realanalysis of the image of greater contact, not by simulation. Anotheroption is for the player to view the image of the bounce and judge aloneor ask for the system to automatically analyse the bounce.

The player may request this automatic analysis and/or display of theselected image, that may be called a “Fox Eye” request, by making a handgesture, which is interpreted by any of the gesture recognition systemsmentioned above, incorporated in the system proposed by the secondaspect of the invention, such as an inertial and/or electromyographicwristband worn by the player, which understands and decodes the handgesture as a preset movement, giving the order to the system to view theimage and/or perform the automatic analysis.

The system proposed by the second aspect of the invention is a system ofmaximum precision (to one millimeter), works with real images, i.e. notsimulated, that, for its embodiments where all the actions are performedautomatically, does not need the intervention of third parties and isinstallable on any tennis court.

Even if the system proposed by the second aspect of the invention isinstalled outdoors, it is not affected by high temperatures on thecourt, nor by shadows, nor by sunlight, nor by people or other objectspresent within the area of coverage of the detection and recognitionsystem, since it “knows” what each detected object is. The size andshape of the spherical element allows the computer means, with the dataof the detection devices, to clearly differentiate it from any otherobject. Neither can any potential intermediate elements between thecamera and the spherical element have a negative influence, since thereare cameras for acquiring images from several different angles, alwaysguaranteeing “clean direct” coverage of the spherical element.

The system does not depend on simulations to be aware of the real bounceof the ball, nor does it require others to manipulate the system norinvolve excessive costs (each high speed camera has a much lower costthan each camera used in the “Hawk-Eye” system).

BRIEF DESCRIPTION OF THE DRAWINGS

The aforementioned and other advantages and characteristics will beunderstood more fully from the following detailed descriptions ofembodiments with reference to the accompanying drawings, to be taken byway of illustration and no limitation, in which:

FIG. 1 is a schematic representation in ground plan view of the systemproposed by the second aspect of the invention with its elementsarranged on a tennis court, and associated with the detection of thebounce of a tennis ball on perimeter lines delimiting the game areas;

FIG. 2 is a schematic representation in elevation view of detectiondevices of the detection and recognition system proposed by the secondaspect of the invention, arranged at different heights relative to aperimeter line of a game area;

FIG. 3 is an analogous view to FIG. 1, but associated with the detectionof the bound of a tennis ball on targets placed in different areas ofthe tennis court, and represented by dashed lines.

DETAILED DESCRIPTION OF AN EMBODIMENT

FIG. 1 illustrates a tennis court with a series of rectangular gameareas Z1-Z10 into which the court is divided, for use during a tennismatch, whereby the rectangular areas Z1-Z10 are delimited by perimeterlines L, which both define the total game area, which for a singlesmatch does not include areas Z7, Z8, Z9 and Z10 but which in doublesmatches are included, as the sub-areas of game, such is the case ofareas Z2, Z3, Z4 and Z5, corresponding to the service areas, it being ofinterest to detect if the tennis ball bounces both on the perimeterlines L of said service areas Z2-Z5 L lines, during the serve, and onthe lines which delimit the total game area, during the rally after theservice.

The system proposed by the second aspect of the invention includes, forthe embodiment illustrated by said FIG. 1:

-   -   image acquisition and registration means that include a        plurality of high speed cameras C, arranged around the tennis        court, preferably at ground level, covering different portions        of perimeter lines L and areas adjacent thereto;    -   means of detection configured and arranged to make an        approximate detection of a bounce of the tennis ball B on one of        the perimeter lines L or proximate thereto, and that comprise a        system of detection and recognition of objects that allows to        discern when the detected object is indeed a tennis ball, and        which includes a series of laser scanners or groupings of laser        scanners S, which together cover all the perimeter lines L;    -   automatic selection means, which in the illustrated embodiment        are implemented in the electronic system SE, in connection with        the detection means, with access to the recorded images, and        configured to automatically select, in response to the        approximate detection, one or more of the recorded images,        including the image corresponding to the point in time        associated with the approximate detection and which includes the        area where the bounce has occurred;    -   means of displaying V to show the selected image to allow it to        be judged by visual analysis; and    -   means of analysis, which in the illustrated embodiment are        implemented in the electronic system SE, and which are        configured to automatically analyse the selected image.

It should be understood that the cameras C are represented schematicallyfor greater clarity. In fact, in particular as regards the cameras Carranged on either side of the centre net N, adjacent thereto, these donot overlap a portion of the lines L such as could be understood by theschematic representation thereof, but instead they are of a size and/orare arranged so that their coverage area also includes the portions ofthe perimeter line immediately adjacent to the respective side of thecentre net N.

Alternatively the arrangement of the cameras adjacent to the net N canbe dispensed with, and instead one or cameras C arranged to one side ofthe court, but at a distance from the net N, can be used, whose angle ofvision allows images to be captured that are behind the net N (through,for example, a translucent portion thereof), in particular images of theperimeter lines L of the other side of the court.

According to one preferred embodiment, one or preferably each of therectangles C do not represent only one camera but instead variouscameras, for example four, stacked one over the other and with differentangles of vision to cover different sections of the same perimeter lineL.

As regards the scanners S, these are preferably arranged at differentheights in order to detect different kinds of objects, including some ata height considered suitable for detecting the players.

The schematic representation of FIG. 1 does not show the connectionsexisting between the different means of the system proposed by thesecond aspect of the invention, either because these are wireless or inorder to provide a greater clarity in the illustration. It should beunderstood that any type of connection between those means is possible.

Also, although FIG. 1 only schematically represents an electronic systemSE, for other embodiments, not illustrated, the system of the secondaspect of the invention comprises the inclusion of one or moreelectronic systems corresponding to one or more of the aforementionedimage recording, selection and analysis means, as well as the controlmeans for the laser scanners L and cameras C.

FIG. 1 also shows a screen as a displaying means V, which displays ascene where the tennis ball B appears deformed when bouncing on aperipheral line L is shown, where said image, if it corresponds to themaximum deformation of the ball B, will be the image that will beautomatically analysed and/or shown to the player or umpire on saidscreen V for its visual inspection.

For a preferred embodiment, each of the elements shown with thereference S in FIG. 1, includes three laser scanners, as illustrated inFIG. 2, one of them S1 arranged at ground level (or very close to it),or first level, and the other two S2, S3 arranged at a certain heightfrom the first scanner, or second level, approximately at 80 cm forscanner S2 and 90 cm for scanner S3 (although these heights may beothers for other embodiments).

FIG. 2 illustrates the trajectory followed by a tennis ball B until inbounces (moving according to arrow D1) on one of the perimeter lines L,and from when it bounces (moving according to arrow D2) thereon, theball being illustrated with a solid line at the time of the bounce andwith dashed line for the rest of its trajectories towards and from thebounce.

Continuing with the description of FIG. 2, during the trajectory of theball B towards the bounce, this is first detected by the laser scannerS3 and immediately after by S2, considering such consecutive detectionsas representing the passage of a moving object, which the detection andrecognition system identifies as tennis ball (as explained in a previoussection). These scanners S2 and S3 operate, according to one embodiment,at a scan speed of 100 MHz, and allow the predefinition of a firstapproximate area or pre-area where the ball B will fall, withdimensions, in ground plan view, in the order of around 50 cm.

Following trajectory D1, the tennis ball B finally enters the coveragearea of the scanner S1, which, according to one embodiment, operates atabout 50 MHz, which, taking into account the average speed at which theball B tends to travel, enables S1 to perform around two or threedetections thereof (the aforementioned approximate detection), at themoment of the bounce and/or just before and/or just after it, within anapproximate area of 15 cm at most, that, while it is relatively smallcannot assure whether or not the bounce occurred on the perimeter lineL, and therefore this approximate detection is used, as alreadyexplained above, to select the image or images captured by the camerasfor the same point or points in time.

As explained in a previous section, the aforementioned predefinition ofthe area in which the ball B will bounce can be used to select and, ifnecessary, activate the camera best positioned to cover this area, andtherefore to perform the mechanical micro-focus thereof if required toensure greater accuracy and speed in the final answer, i.e. in theresult of the analysis of whether the ball B has bounced inside oroutside the game area.

All these actions happen in milliseconds and time is crucial to activatethe camera and mechanically focus on the area of the bounce, ifnecessary, and therefore, to be able to perform the analysis of thechosen image almost in real-time with high definition without usingmega-cameras, i.e. the very expensive cameras of the “Hawkeye” system.

The scanner S1, in its various detection points, before the bounce,during the bounce and right after, provides the approximate area of thebounce in a much more accurate manner than the area predefined by S2 andS3, but above all it gives information about the moment of the bounceand the adjacent moments that allow the system and its choice to beoptimised to be able to work with almost in real-time with theinformation from the selected high speed camera, thus achieving, thanksto the mechanical focus, maximum errors of half a millimeter comparedwith the 3 millimeters of the “Hawkeye” system.

FIG. 3 shows the system proposed by the second aspect of the invention,incorporating the same elements as the system illustrated by FIG. 1, butapplied to the detection of the bounce of the tennis ball B on sometargets A1-A8 distributed throughout the court, within the game areas,in particular they have been illustrated within areas Z1 to Z6, or evenwithin two areas, as is the case of the target A4 with respect to areasZ4 and Z6. Obviously, the illustration is schematic only, and a singlemovable target may be used from different positions.

According to one embodiment, targets A1-A8 will not be visible on thetennis court itself, but they will be virtual targets (or a singlemovable target occupying the different positions illustrated) visible ona screen that shows them in relation to the tennis court, i.e. to show,for example, a tennis court with the targets as shown in FIG. 3, or apart thereof (for example those placed on the side on which the playershould aim the ball).

Also, although FIG. 1 only schematically represents an electronic systemSE, for other embodiments, not illustrated, the system of the secondaspect of the invention comprises the inclusion in one or moreelectronic systems of one or more of the aforementioned image recording,selection and analysis means, as well as of control means for the laserscanners L and cameras C.

FIG. 3 also shows the electronic system SE and a screen as a displayingmeans V, which displays a scene where the tennis ball B appears deformedwhen bouncing on one of the targets, specifically on target A1, whichwill be the image that will be automatically analysed and/or shown tothe player or umpire on said screen V for its visual inspection.

The embodiments described with reference to FIG. 1 are also valid todescribe analogous embodiments with reference to FIG. 3, simply bychanging the component on which the bounce of the ball B occurs, that inFIG. 1 is the perimeter line L and in FIG. 3 is the target A1-A8(obviously the same can be said for another kind of component than arenot either the perimeter lines L or the targets A1-A8).

A person skilled in the art could introduce changes and modifications inthe described embodiments without departing from the scope of theinvention as defined in the attached claims.

The invention claimed is:
 1. A method for determining whether aspherical element impacts with a playing field, or with a componentarranged on said playing field or proximate to said playing field, saidcomponent comprising one of a physical structure, physical marking, orvirtual marking, and said playing field being made up of game areas,where the method comprises the steps of: a) acquiring, with detectiondevices comprising at least a plurality of high speed cameras, asequence of images of at least one area of surveillance of said playingfield covering at least part of at least said component and storing thesequence of images on a memory; b) performing, by a processor of acomputer system operatively connected with said detection devices, whenthe detected object is indeed a spherical element, an approximatedetection of the impact of the spherical element with said component orproximate thereto; c) automatically selecting, by the processor inresponse to said approximate detection, at least one image, from saidsequence of images, acquired for a point in time that includes the atleast one area of surveillance of the playing field where a bounce takesplace and when the bounce takes place; and d) determining, with theprocessor, that the spherical element has impacted with said component;wherein: prior to said approximate detection of the impact of step b), apre-detection of the passage of the spherical element through at leasttwo points located at different heights is performed with at least thedetection devices, and based on said pre-detection, a prediction of thetrajectory to be followed by the spherical element towards the groundand a predetermination or predefinition of an approximate area where theimpact or bounce will occur is performed by the processor, saidpre-detection being performed using at least two detection devices. 2.The method according to claim 1, wherein said detection and saidpre-detection of the impact are done by the same plurality of high speedcameras.
 3. The method according to claim 1, wherein said detectionmeans further include sound detection devices which are operativelyconnected to said processor.
 4. The method according to claim 3 whereinsaid prediction of the passage of the spherical element through at leasttwo points located at different heights is further performed with theassistance of a three-dimensional type object detection and recognitionsystem including at least scanners located at different heights.
 5. Themethod according to claim 3, wherein the scanners are laser scanners,and the three dimensional type object detection and recognition systemfurther comprises ultrasound devices, and the method comprisesperforming said automatic discerning using said laser scanners to obtaindistance information, with respect to said laser scanners, from eachpoint of contact of the laser beams emitted by the laser scanners withthe detected object, and based on said distance information obtained fora group of contact points.
 6. The method according to claim 1 wherein insaid step a) said acquisition of images only covers several targetswithin the area of surveillance of said playing field.
 7. The methodaccording to claim 1, wherein said approximate detection of step b)comprises performing at least two detections that include at least amoment when said impact or bounce happens and/or at least a momentimmediately before the impact or bounce and/or at least a momentimmediately after the impact or bounce, using in said step c), as thetime point for the automatic selection of said at least one image, atleast one of the time points of said at least two detections.
 8. Themethod according to claim 1, wherein said step d) comprises showing saidat least one selected image on displaying means to allow performing theaforementioned analysis by visually judging the image, or to supplementsaid analysis.
 9. The method according to claim 1, wherein: said step c)comprises selecting a plurality of images from said sequence of imagesand arranged in sequence, wherein the plurality of selected imagesincludes the at least one image acquired for the point in time of theapproximate detection and images for points in time before and afterthat point; and said step d) comprises a video playback including saidplurality of images, in slow motion forward and/or backwards and/orstopping in order to perform the aforementioned visual judgment.
 10. Themethod according to claim 1, comprising performing said step a) theplurality of high speed cameras covering the various components of theplaying field, or arranged on or proximate thereto, from different sidesthereof, and/or the different sections of the perimeter lines from bothsides and/or ends thereof and/or all the targets from different sidesthereof.
 11. The method according to claim 1, comprising using saidpredefinition of said approximate area where the bounce will occur toactivate and/or control one or more high-speed cameras whose coveragearea includes the approximate predefined area.
 12. The method accordingto claim 1, comprising automatically discerning when the detected objectis indeed a spherical element based on the difference in the detectionsignals caused by being detected by the detection and recognition systemin comparison with the detection signals resulting from the detection ofother elements, stationary or mobile, of at least different shape andsize.
 13. The method according to claim 1, wherein said sphericalelement is a tennis ball and said playing field is a tennis court, withsaid game areas being the different rectangular areas of said tenniscourt.
 14. The method according to claim 1, wherein said sphericalelement is a tennis ball and said playing field is a tennis court, withsaid game areas being the different rectangular areas of said tenniscourt and wherein the method comprises selecting as perimeter lines onwhich check when the tennis ball has really bounced or not, thefollowing: those demarcating a respective service area to which a playerwill serve or has served, to judge whether the tennis ball bounces in orout of the service area; or those demarcating the total game area for asingle or a doubles match during the rally after the service.
 15. Themethod according to claim 14, comprising receiving information onwhether, after the service, the tennis ball has touched a net or not ofthe playing field, and based on this information, after judging whetherthe tennis ball bounces in or out of the service area, the methodcomprises issuing a signal indicating that: the serve must be repeated,when the ball has bounced inside the service area but after touching thenet, or the serve must not be repeated, when the ball has bouncedoutside the service area after touching the net.